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ABSTRACT ^ 

The purpose of this paper is, generally, to discuss 
issues related to the, evaluation of implementation and, specifically, 
to provide practical advice about Follow Through implementation and 
its evaluation. First, the discussion explores philosophical and 
theoretical assumptions regarding (1) the nature of implementation 
and evaluation; (2) realities of implementation as a unique form of 
socialization; and (3) approaches to evaluation that are compatible 
with the specified realities of implementation. Non-mechanistic, 
non-reductionistic philosophical assumptions and consonant 
theoretical positions are favored. Haxt, a discussion of realities of 
implementation provides a perspecti^ 1=! on Follow Through evaluation 
efforts, analyzes implementation a^ socialization, and offers 
recommendations concerning the future of Follow Through. The 
concluding discussion of evaluation offers a concrete proposal for 
the evaluation of implementation. While many compatible evaluation 
stategies are listed, the judicial evaluation model is seen as the 
single approach most compatible with implementation realities, and a 
combined documenting and judicial evaluation approach is recommended. 
Briefly addressed are the = inferential dimension of educational ' 
measures and the importance of Follow Through model sponsors. 
Appended is a rationale for a national Follow Through database that 
specifies needed categories of information and evaluation/research 
strategies. Also appended is a list of elements of child data and 
site descriptor computer files to be maintained at each Follow 
Through location. (RH) 
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CHARTING THE COURSE OF l^^^^^imji 



I. INTRODUCTION 



By placing this paper's discussion »J Impffiafv^^tatio' ■r}^^<-in thft 
framework of "charting a course," it is^ t-ntrindea that the ftv?t3 ba 
placed from the very outset on a few ess^-'itia? t^rr .^s analogou-^ 
charting situations. These themes have ^s"Z£r, convictions W^z 

the following: (a) innovative program f&s^mn,l!r-t^on in educ<- :,f,'n 
is in laj-ge part a matter of being able to ff-^^. t-ne's way thrc rc-V a 
complex seH&s.of mazes - including tiva. <v,*?:.Kt1c?!S of pers&^ - 
surrounding the real world of the schools, ^'li the iiathways "hr.;,!^,*. 
tiiese mazes are in reality merely passable- routes .bfc^-aen thD 1. x«r^ 
weaving motions of several layers of systems; routes that are onl.y 
really open to er.perience and understanding, and (c) the whole journey 
through this system-maze is best captured by thorough, alert 
documentation. 

To further set the stage for the discussion, and even more 
graphically try to convey the appropriateness of the notion of "charting" 
for dealing with this issue. It is perhaps useful to picture the whole 
innovation/implementation process as having some 'basic similarities with 
the attempt to produce sorely needed. rain by seeding clouds with silver^ 
iodide. The seeding operation represents a carefully weighed decision 
to intervene in the continuous, massive movement of weather systems as 
they roll across the earth's- surface. The decision to seed, furthermore, 
is made when rs ..jnsidered absolutely ess^; 'O ame^ a 
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drought situation, and only after carpfi.i .i- ^ u . 

y arier careful study beforehand, and 'ollow- 

up tracking afterwards, of the - synopfc chart. u - • 

•'^ ^"^"^^ - which idsntify and 

locate the complex patterns of high and low bressure sv.tpmc . 

pressure systems, temperature 

and humidity levels o-^-Fori-r u 

y ievels. e.fects or the prevalliag winds at varioas 

altitudes, etc.. which mke up what we call weather. 

V. ™ain polhts-.to be .ade, of course.,, „ using this analogy, are 
t^at educational interventions, and stud, of their i.pl«.„tation. are 

the etherise steady flow of education's complex patterns, and then. on 
* has,s Of carefu, p.. during, and post charting routines, introducing 
the projected re^dy at so.e fairly precise point, in son« well-selected 
unit of education's huge, intricate system. 

It .ight be further noted, in tenns of appropriateness for 
-pigmentation issues, t^t the weather analogy is likewise a clear 
binder of at least two other points: - the primitive state of our 
knowledge of intervention agents- in hoth areas - cloud-seedina is at 
best a crude and awkward process, with a lot of hit and miss character- 
-tics. and our know-how with regard to educational interventions 
despite al, recent attempts, cannot honestly be considered to be much 

beyond the level of rain^king expertise, and (b, the notoHous 
fallibility Of weather predictions, with educational pi^dicting a not 
~ch ™,re. accurate cogigrpart. On the limping side of the analogy 
n IS of course, crucial to point out that the inani^te elements 
comprising the complicated' web of interactions found in weather systems 

are far outdone in complexity hv tho c^i* l • 

urapiexiry .ny the self-knowing, abstract tfhinking 
beings we have to d( ^' j 

.lavB LD a. t:he educational sphe- 



All of the above is not said by way of promoting discouragement, but 
only by way of setting what this paper considers a realistic background 
for taking up the charges assigned to it. These charges are essentially 
to discuss issue's related to evaluating implementation, and to provide 
sane practical considerations regarding implementation, and its evaluation 
as part of Strand Two (Search for Future Approaches) of NIE's efforts to 
provide redirection for Follow Through, The paper's response to these 
charges, keeping in mind the complex systems theme, and the weather analogue 
introduced earlier, will be In the form of three major sections organized 
as follows: 

1. Philosophical/Theoretical Asstmptions Underlying this' Discussion. 
Philosophical presuppositions very clearly affect one's outlook 
on the nature of implanentation and evaluation, and consequently 
one!s approach to evaluating Implementation. This paper favors 
non-mechanistic, non-reductionistic philosophical assumptions - 
and theoretical positions consonant with this stance - because 
they appear to be more consistently meaningful in attempting 

to understand everyday educational reality. 

2. Implementation Realities* 

^A close look at implementation phenomena in the real world of 
education leads one to concludii that it i*- a unique form pf 
socialization related to restructuring the perceptions, roles 
and actions of educating individuals functioning within at least 
four plausible modes of existing organizational structure (system), 
whil<> allowing that mutual adaptation of innovation/existing system 
is always at least minimally involved. 



3. Evaluation Compatible with Implementation Realities; Recoinnendations. 
Not only is evaluation felt to be in need of a non-reductionistic 
base to be able to effectively engage in educational inquiry in 
general, but this is especially held to be the case in attempting 
to handle implementation phenomena with their continuously 
evolving process characteristics, i.e., complex htmn changes 
occurring against a backdrop of existing complex human crganiza-^ 
tions/systems. 

The evaluation approaches that rely primarily on straightforward, 
comprehensive documentation, and those developed in analogy with 
judicial/quasi-judicial processes appear to be particularly 
appropriate for dealing with implementation because, on the 
one hand, . they give prominence to thorough, continuous 
description, which is presently our best means of keeping track 
of the continuously shifting configurations of implementation 
"in vivo," and, on the other hand, feature, non-mechanical measures, 
i.e., well-reasoned judgments,, to cope with educational reality, ' 
wnich is perhaps never more fluid, and less susceptible to fixed 
measures than when implementation is ths^ focus. 

Given the above premii*>es, i*e^, having the responsibility of 
accounting for the iimpl/anentation process, with its primary 
characteristics of c«p'lex human changes being introduced into 
existing complex htm^ svsterrK, a sensible, logical strategy 
would seem to he a ioiCit, interdependent use of comprehensive 
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documentation and quasi-judicial reviews/hearings, with the 
former serving as an ongoing feeder system for the latter, v/hich 
would occur at regular intervals to synthesize and provide 
direction for each succeeding stage of implementation, and/or 
yield recommendations regarding the advisability of continuing 
any present attempt in its existing form. 

PHILOSOPHICAL AND THEORETICAL ASSUMPTIONS REGARDING THE NATURE OF 
IMPLEMENTATION AND EVALUATION. 

The paper makes several philosophical and theoretical assuir^ptions 
about the unique capabilities of human beings, the participants in 
implementation, and, by implication, about the nature of implementation 
which, in this conception necessarily involves introducing change in 
interaction with these unique.capabil ities: 

(1) Human beings are persons, with characteristics that 

distinguish them qualitatively from other living things; 
among a number of essential characteristics are special 
cogr . ../^ capaoilities. that allow humans to be active 
knowers who know that they know, i.e., who are capable of 
' thinking, of conscious planning and rule formulation based 
on the ability to engage in internal structuring and 
modify ing of concepts. 



(2) Given the above, the process of innovation/implementation 
necessarily implies, at least to some extent, having the 
persons involved change their thinking as regards a particular 
area of the educational process, 

(3) Furthermore, the persons participating in innovation/ 
implementation must be dealt with not only as separate 
individuals, but as already coninitted participants - 

again, to at least some degree - in an existing organizational 
structure (system), which defines current educational reality, 
in varying degrees, for these individuals- 

The paper aUo makes assumptions about the nature of educational 
inquiry/educational evaluation, and hence about the nature of 
evaluation as applied to implementation. Again these assumptions 
rely on iion-reductionistic views on the nature of human knowledge and 
educational phenomena. 

A. N^ t'urg 2l HU'fl^ Beings/Human Knowing 

In making the above assumptions, the paper is clearly sic^ng 
with non-positivistic, non-associatlonistic vttws of hionans and 
huniar) knowing. Positivism, which has tended to govern, either 
formally or informally, the thinking of educational psychologist;5 
for most of this century, has always stood for an all -encompassing 
reductl'^nism, i.e., there is nothing radically different about-the 
way hunan beings know and learn, as distinct from other living 
things. Furthermore, in this view, especially 1n the form of 
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Behaviorism, i-t is also commonly held that relatively few, 
relatively simple laws relating neuronal excitation to response 
characteristics will suffice to explain what are usually 
considered unique human capabilities* This is particularly the 
case if these simple eleirents can be viewed as gradually com- 
bining via, again relatively few, relatively simple laws regulating 
their progressively more complex associa^'on. 

The position endorsed by this paper does not consider 
Positivism or Assoclationism tenable philosophical positions in 
the face of everyday reality, perhaps especially everyday school 
reality. The mechanistic, >-eductionistic principles on which 
they rely are simply lacking. m adequacy 1n ^h^- fp :e of vyhat are 
most characteristically human activities » , rticularly as regards 
human capability to function as purposive, self-knowing p/er^ons*: 
With Wittrock (1974) and McNamara (1980) the pap^j* believes it 
is time" to recognize that we have been too- long under the sway 
of Positlvistic convictions, and to move on to a more balanced 
outlook on human beings and the ways in which they know. <^ At the 
same time it is important to recognize the contributions of 
Positivism and A5:^ociationism, especially their preoccupation with 
clear thinking and specification, and avoidance of tendencies to 
rely on convenient, catch-all phrases that have not been subjected 
to rigorous analysis. It is important also to acknowledge the fact 
that there are undeniably some limited aspects of human behavior 
that can be adequately explained in mechanistic, reductionistic 

■ -7- - 
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terms, but these aspects do nof'cx^inprise the full range of human 
capa6ilTties^» and, furthermore, it is only positions that can 
account for this full range which can legitimately, because 
knowingly, treat hunans temporarily as if purely mechanical beings. 
This temporary process is a directly intended abstraction involving 
conscious disregard of the full complex of human capability for a 
time in order to treat humans in, e- g., purely quantitative 
terms to describe certain limited aspects of human phenomena. 

B. Nature of Innovation/Implementation 

1. Individuals Must Undergo Conceptual Changes. 

There are some clear cut implications for conceptualizing the 
nature of educational innovation/implementation in making the above 
assumptions. The iriost fundamental implication is that introducing t 
innovation, and then seeing to its implementation, clearly involves 
conceptual changes In some senise on the part of participants. 
Human beings in this view. I.e., persons who know that they know 
and actively monitor and regulate this knowing process, especially 
when these human beings are themselves engaged in the daily process 
of teaching others. I.e., introducing conceptual change in 
systematic fashiorj^^to students, cannot be expected to abrogate 
this special human capability and respond in purely mechanistic 
terms when they themselves are the participants in an educational 
change process. 

The thinking of Strike and Posner (1977) seems especially 
pertinent for further understanding the nature of the conceptual 




change to be undergone by individuals according to the above views, 
and even -more pertinent for understanding conceptual change in 
the context of curriculum organization, which must, by definition, 
underlie most proposed innovations/implementations. They support 
the following views of conceptual change — patterned after 
Toulmin's account of scientific procedure — as one of the most 
realistic: 

Conceptual change is evolutionary, not revolutionary. .. 
An evolutionary model of change allows us to evaluate any 
particular conceptual change against an enduring background 
of rational standards and explanatory ideals. Conceptual 
change is not a matter of judging between comprehensive 
views but rather of judging concepts in relation to other 
concepts, with respect to their adequacy in interpreting 
problems and fulfilling explanatory ideals... Concepts 
adapt to art intellectual environment much as organisms 
adapt to a biological environment. 

* (Strike and Posner, 1977, p. Ill) 

In subscribing to this outlook on conceptual change. Strike 
and Posner simultaneously endorse non-mechanistic positions regard- 
ing concept formation, cognitive structure, and thinking, proposing, 
in fact, that, as regards curriculum organization^ "cognitive 
status and processes can be regarded as the *eirt}odiments' of the 
kinds of logical and conceptual features which characterize 
organized bodies of propositions" ^op.cit., p. 136). 

Similar convictions about the nv'^cessity for conceptual change 
in implementation have characterized the work of G. E. Hall and 
his colleagues at the Research and Development Center for. Teacher 
Education at the University of Texas at Austin. Lolicks and Pratt 
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(1979) describe tha application of this group's "Concems-B^sed 
Adoption VIodeV in the Jefferson County, Colorado schools. This 
approach makes the following four assumptions: 

1. Change is a process, not an event, 

2. Change is accomplished by individuals, not institutions. 

3. Change is a highly personal experience- 

4. Change entails developmental growth in both feelings 
about and skills in using new programs, 

(Loucks and Pratt, 1979, p. 213) 

McLaughlin (1978) has also concluded from study of both 
organizational change projects as well as general change - agent 
efforts that the primary focus in innovation and implonentation 
must be on individual participants, and that mutual adaptation 
is essential. "The evidence we have seen strongly suggests that 
the developmental process of mutual adaptation Is the best way tc^ 
ensure that change efforts are not superficial, trivial, or transit- 
ory" (McLaughlin, 1978, p. 31). Mann (1978c) has extended this 
conclusion to the proposal that innovations take the form of 
"user-driven systems" in which individual teachers are given special 
Incentive pay to raise the scores of targeted student groups. 

2. The Individuals to Be Changed are Part of an Organization 
to be Changed 

Though the above conronts stress that individuals must 
change in Innovations, and, therefore, ^mply that this paper 
is in clear-cut agreement with the Concerns-Based tenet; 
cited earlier, that "Change is accomplished by ..individuals," 




there is simultanGous reluctance to accept the second part 
of that tenet, i.e., "Change ,is not Accomplished by 
institutions," without some distinctirms. It is felt that 
until the day arrives - and perhaps it would be a good thing 
if it does — when a proposal like Mann's, also cited above, 
regarding direct incentives to teachers becomes a reality, 
(and perhaps even then, only in reduced form), it is only 
realistic to recognize that innovations/Implementations presently 
involve individual teachers £ua members of organizations 
(institutions), aiid that this organizational membership dimension 
must be taken into account in conceptualizing all that is 
important to implementation, reality. It is in this context that 
Elmore's (1978) insights - coupled with the realistic approach 
of symbolic i^iteragtionism, to be discussed later - seem to 
offer a useful means of handling the organizational dimension. 
He has concluded, along with many others, that the widespread 
record of unsuccessful innovations is principally attributable 
to failures in implementation, and then draws the further 
conclusion that "implementation, failures are failures of organization • 
Understanding organizations, therefore, is crucial in, Elmore's think- 
ing, if irriplemention Is to be handled more successfully: 

I will develop four organizational models representing 
what I see as the major schools of thought that can be 
brought to, bear on the implementation problem.. The ^ 
systems ma nagement ^model is my attempt to capture the 
orgamzatTonal assumptions underlying the mainstream, 
rationalist tradition of policy analysis,.. The bureau- 
cratic process model represents. the sociological view 



of organizations, updated to include recent research 
by students of 'street-level bureaucracy'... The 
organizational development model represents a relatively 
recent combination of sociological and psychological 
theory that focuses on the conflict between the needs of 
individuals and the demands of organizational life. 
Finally, the conflict and bargaining model represents a 
tradition of inquiry that addresses the problem of how 
people with divergent interests coalesce around a 
coninon task. 

(Elmore, 1978, pp. 188-189) 



After extensive elaboration of the models, Elmore reaffirms .their 
usefulness for "certain commonsense explanations as to why social 
programs fail and, in addition, feels that not only is there presently 
no conceivable way to reconcile these four different viewpoints, but 
this diversity allows different aspects of implementation problems to 
'be highlighted and solutions formulated accordingly. 

Kirst and Jung (1980) see the typeof analysis proposed by 
Elmore as part of a second stage of theory development, following 
on an initial, descriptive,, short- run case study phase, in attempting ~ 
to explain the workings of implementation. They stress the timeliness 
of capitalizing on the advances made in stage two studies-like Elmore's 
and perhaps ushering in a third phase by focusing on a longitudinal 
case study approach. They illustrate the approach by following the 
course of Title I implementation ^over 13 years and noting the changing 
influence of various interest groups on interpretations of implementa- 
tion, as well as. the continuous increase in the power of USOE officials 
to impose strict interpretations of compliance. Farrar, De Sanctis . 
and Cohen {'1980) would also argue for the importance of a longitudinal 
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perspective from a slightly different point of view, proposing an 
evolutionary connatation for implementation consonant with^Strike 
and Posner's (1977) statements above about the evolutionary nature of 
conceptual change: ' • 

Implementation is not the carrying out of a 
formulated policy but part of its evolution. 
And in that evolution — in certain, circumstances 
or with certain programs - a multitude of local 
dispositions and actions are more important than 
the dispositions and actions of federal agencies. 

(Farrar, De Sanctis and Cohen, 1980, p. 167) 

It is perhaps useful to conclude these initial considerations on the 
nature of innovation/implementation by referring to Mitchell and Spady's 
(1978) article on the necessity of broader change movements, (but with 
implications for all. innovations), such as the recent push for 
"competency based education," having to be viewed in an organizational 
context also, though in this case the organizational context has to do 
with outcome expectations that analysis would associate with four 
alternate bases of education, i. e., competency, development, social 
integration, and social responsibility, 

C. Nature of Educational IriquiYy/Evaluation 

In the earlier part of this discussion, when the focus was on 
basic aspects of human nature, human knowing, and conceptual change,* 
much has already been said t hat would be a p plicable he re, _i.e., the 
same rejection of reductionistic and mechanistic principles would 
be repeated. It only remains to^ summarize some statements by a 
number of major writers who have dealt more explicity with educational 
inquiry and educational evaluation, while simultaneously stressing the 
inadequacies of positivistic and similar assumptions. 



First of all it is important that the nature of educational 
phenomena be clearly established under this viewpoint, Gowin (1972) 
makes a very precise statement: 

•••a most significant fact about educational 
phenomena: They are man-made (artifactual J., 
not natural. They are therefore not likely 
to yield laws and other modes of invariance 
such as the natural sciences report in that 
domain. Whatever regularities researchers 
are to find in educational phenomena will have 
been determined by human beings in a social 
context, (Gowin, 1972, pp. 9-10) 



Sarason (1978) reminds us that in social matters, and, therefore,- 
equally so in education there are no "once-and-for-all solutions in 
the scientific sense"; essentially the same problems have to be 
solved over and over again, Scriven (1980) makes much the same 
comment, while proposing that educational inquiry shed once and 
for all the sense of dependence on, of derivation from, of 
application via the social and behavioral sciences," For Scriven: 

Language is a better model of complex human behavior 
than the motion of small particles or large planets. 
We understand language; we understand many unpredictable 
and unprecedented linguistic events; we often understand 
why language is used. Yet we know no exact or statistical 
rules or laws of language. There are patterns in it; but 
not the precise patterns of planets or the probability 
distributions of particles. These are the patterns we 
must learn to findand describe in the study of teaching 
and administration and learning, 

(Scriven, 1980, p. 30) 

House (1978) provided a taxonomy of major evaluation models, 
relating each to the philosophy of Liberalism; his major work on the 
"Logic of Evaluative Argument" (1977), however; seems to reveal his 



-14- . . 



16 



preferences for models with non-positivistic bases, stressing, e.g., 
the continuous importance of "qualitative argument." !n a very 
recent statement (House, 1980), he goes so far as to say that 
explanatory frameworks shift with changing social conditions: 

...social science perspectives generally, describe or 
explain a social world which is itself constantly changing. 
At one time, a technological perspective may better 
explain the social reality, but at a later time the 
political perspective may do so. For example, as 
society becomes Increasingly fragmented, political 
and cultural perspectives seem more salient as 
explanatory frameworks. (House, 1980, p. 3) 



Guba (1980) and Sanders (1980) both argue convincingly against 
positivistic assumptions. Guba stresses the need for naturalistic 
versus conventional inquiry, while allowing that the "quantitative 
postures" of the conventional approaches are just as acceptable 
in naturalistic investigations, but under different basic 
assumptions.. Sanders proposes a "developmental research" approach 
that recognizes ^;he sui genesis nature and complexity of educational 
reality. 

Strike (1972) stressed the need for Verstehen in educational 
inquiry: - 

Man, for Weber, is a purposeful, rational, and 
value-seeking thing. Man has a subjective life. 
However, man's behavior can be accounted for 
scientifically by the simple expedient of^ incorporating 
man's subjective life into an account of his behavior. . 

(Strike, 1972, p. 45) 

In a statement conceptualizing the practical science naturie of 
educational inquiry (Strike, 1979), he emphasized that education 



is unavoidably engaged in the business of "normative and ends - 
means proposition.*;," as well as "factual" and "theoretical" 
propositions. 

Stake (Page and Stake, 1979) feels evaluation has nothing to fear 
from being "snore subjective," and more recently (Stake, 1980) makes 
the point that it is an important role for researchers (and 
evaluators) to serve as facilitators for a "user's naturalistic 
generalizations," I.e., those "derived from tacit, personal, 
experiential learnings," * 

The above is only a brief summary of the assumptions/conclusions 
' that will guide tlie discussion of evaluating implementation in the 
concluding section. As a prelude to that section, however, it seems 
best to further expand on soira of implementation's realities, since 
evciluation of this phenomenon has to be particularly aware of and 
sensitive to all the continuously evolving aspects of implementation. 

T>K ir!PL£hjHTAriON REALITIES < 

Ttjt' .^^recijoi.jc! consideration of innovation/implementaition was on a 
br^:i:v:;^>^ prii ^?:>oph^::t:>l/theoretical plane to provide the setting for 
thi^ ScCtiort which attempts to make the issue more concrete and 
ivnenable to explicit evaluation activities, as well as to prepare the 
way for some explicit recommendations regarding sponsor models/ 
implementation. The section will be divided into three main parts; 
in the first the attempt will be made to put Follow Through implementa- 
. tion efforts into perspective by applying Elmore's (1978) models; 
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in the second part attemptins to analyze implementation as socializa- 
tion, , using Fullan's (1980) insights on this subject, as well as his 
delineation of implementation factors, and in the third part formulating 
some recomnendations regarding the future of Follow Through based on 
these implementation considerations. ^ 

A. Follow Through Implementation in Perspective 

^ House (1979) took the point of view that Follow Through was a. 

child of its. time in many respects, not the least of which was the 

fact that the Systems Analysis approach (a favorite of USQE and 

other federal agencies at the time) , gradually took more and more 

solid hold of both program direction and the national"^ evaluation 

during the program's first ten years. Ine Systems Analysis approach 

is one form of Elmore's (1978) Systems Management Model (Model I) 

and almost every point he makes in describing the characteristics 

of this model provide an equally accurate description of many of 

the implementation tribulations of Follow Through - and also what 

seems to have been some Of the prime assumptions about the nature 

of implementation in USOE's mind, the sponsor's mind, and sites' 

mind, as can be seen in, e.g., Haney's (1977a, 1977b), Kennedy's 
(1977, 1978), Wisler, Burns and Iwamoto's (1978), arid Hodges 

et al.'s (1980) discussions of this issue. This was perhaps 

inevitable, given the fact that, in Full an and Pomfrefs (1977) ' 

terms, the "fidelity" - in contrast to the "mutual adaptation" - 

conception of implementation was apparently taken for granted - or 

was the exclusive choice, perhaps, in some cases — by seemingly 

all concerned. It should also be noted that dur;ing most, if not all 



cf the early years of Follow Through, the spirit — If not the 
letter — of positivism still exercised considerable influence 
over conviction* regarding educational research and evaluation, 
not to mention over educational theories that kept substantial 
ties to early behaviorism, and, as already mentioned, over 
prevailing convictions about implementation, even if only in 
once - removed fomit e.g., via Systems Analysis. It seems to 
be fair to all concerned to say that Elmore's (1978) Model I 
(Systems Management) and Model II (Bureaucratic Process) go far 
in explaining practically every aspect of the direct program 
implementation ccnvictions and operations that have existed 
througiiout most of Follow Through up to this point. It seei^ts^ i 
fact, not to be overstating the case to see some analogies between 
a widespread outlook on what should constitute Follow Through Model 
implementation, at least during the program's early years, and 
what Goodenough (1966) and other anthropologists have referred to__ 
as "cargo cult" expectations. As will be discussed below, there 
was some relationship between Follow Through and Elmore's. Model IV 
(the Conflict and Bargaining Model), but apart from large city 
situations, where powerful Unions had to be dealt with. Model IV 
came Into play primarily in,. relation to survival rather than 
implementation per se. 

Elmore (1978) has indicated that within the framework of 
Model I (Systems Management) implementation failures are logically 
attributable to "bad management," i.e., no specifying goals clearly. 



not assigning/monitoring responsibility carefully enough, etc. 
Furthermore, there is a normative dynamic to the model which 
dictates that, if implementation ia failing, it is not the fault 
of the Systems Management Model - since things ought to be this 
way - so the only recourse is to restructure the situation to 
fit the model, (Elmore feels the model works best in private 
business settings where the kind of control the model calls for 
can, for the most part, be exercised is a matter of course.) 
For Model II, the Bureaucratic Process Model , implementation 
problems are seen as principally a matter of failing to Influence 
existing organizational routines, so that as a result, they inter- 
fere with the innovation's operations. This model's normative side 
does not have the force of Model I and simply consists in stressing 
the importance of somehow influencing the routines that might be 
problemmatic. The history of Follow Through and its handling of 
the "implementation problem" seems well captured within the boundrii 
of these two models, for the most part, and principally within 
Model I. - 

It is in order at this point to explain the comnent made above 
about Model IV (Conflict and Bargaining! and Follow Through- This 
connection seems best explained — again, apart from large city 
Union problems — in saying that Model IV appeared to enter the 
scene not very much in relation to implementation-perseTi)ut-rather 
in relation to survival. Parents from sites continuously and 
vigorously lobbied for the program's survival, but in itself ' . 



implementation was not the foremost Issue in the lobbying effort. 
Model IV could also be said to have characterized Sponsor-^USOE 
relationships after a time, but again survival , rather than 
implementation appears to have been the uppermost consideration - 
though perhaps implementation figured here also, but under the 
form of what Haney (1977a) calls the "Penelope Ploy," i.e., the 
continuous postponement of the effort at final implementation 
specifications, etc., so that effectiveness considerations - 
whether consciously or unconsciously - would not have to be 
dealt with in clear-cut terms. 

To return to the central theme of the present discussion, 
however, i.e.. Follow Through Implementation efforts as covered 
by Elmore's (1978) Models I and II, it is Important to note that 
in taking the above position this paper is not trying to detract 
from what Kollow Through has accomplished in the implementation 
sphere. With Haney. (1977a/l 977b) , Wisler, Burns and Iwamoto 
(1978), and Hodges et al. (1980), the paper has great appreciation 
and admiration for the program's being able to obtain any successes 
at all in the face of having to operate without a clear definition 
of purpose; "unplanned" Planned Variation, in many respects; 
constant shifting of the ground rules, so to speak; etc. There 
is likewise great appreciation for what Follow Through Sponsors 
have accomplished, despite all odds , in pioneering a new third 
party role, as documented by not only the writers mentioned above, 
but also., e.g., by Kennedy (1977, 1978), Nero and Associates (1975), 



and Rath (1976) • But, as is evident from the preceding section 
on philosophical/theoretical cor^si derations, the paper would 
naturally have to be snore interested in Follow Through relationships 
with Elmore's (1578) Model III (Organizational Development), than 
his other models; those relationships, however, seem tenuous at 
best, if not altogether nonexistent. The papiar logically would 
have to see broad normative influence" of Model III as the ideal 
situation. 



Model III posits that failures in implementation "are not the 
result of poor management control or the persistence of bureaucratic 
routines, but arise out' of a lack of consensus and commitment among 
implementers" (Elmore, 1978, p, 209), In Elmore's thinking, Qven 
the idea of "mutual adaptation" does not completely explain all 
that is meant by "organ'liational development," since: 

The real significance, ..is that it effectively turns the 
entire implementation process on its head. It reverses 
' what we instinctively regard as the 'normal' flow of 
policy, from top to bottom., .The result is that, in terms 
of the effective structure of organizations, the process 
of initiating and implementing new policy actually begins 
at the bottom and ends at the top ,/, The role of those at 
tRe top of the system, tRen, is necessarily residual,, • 

If one accepts this view, the important business of 
implementation consists not of developing progressively 
more sophisticated techniques for managing subordinates' 
behavior but enhancing the self-starting capacity of the 
smallest unit, (Elmore, 1978, pp, 21T-212) 



It is one of the main convictions of this paper that not only 
is it time for Follow Through to move consistently in this direction, 
but that the program will be stunted in its natural development if 
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It does not — if it is even allowed to continue to exist as 
before. More will be said on this point in the recommendation 
discussion below. Before making the recommendation, it seems 
useful to further expand on implementation realities by pursuing 
\he concept of implementation as socialization with Fullan (1980). 

B. Implementation as Socialization 

According to Fullan (1980), implementation is a socialization 

process with varying degrees of interaction among personnel, and 

between personnel and materials, strategies and overall philosophical 

commitment. There seems to be little doubt that framing imp! ementati 

in a socialization context goes a long way toward concretely 

characterizing what has usually taken place in implementation 

efforts, and that this approach can complem'^nt Elmore's (1978) 

insigWs especially where Model I (Systems Management) and Modal 11 

(Bureaucratic Process) seem to be the ruling Influences. 

Implementation as socialization seems less applicable to Model IV 
(Conflict and Bargaining), and to Model, III (Organizational 

Development), though, of course, relevant to some degree here also. 

Symbolic interactionist viewpoints like those of Becker (1968) 

and culture and personality perspectives like those of Spindler 

(1968), with their emphases on discovering/maintaining identities 

within socialization situations njight well be able to reconcile 

any apparent discrepancies between at least Model III and 

"socialization." Mischel's (1979) recent work on "cognitive 

economics" might also provide some light on how disparate 



Individual/societal (Implementational) demands might be 
harmonized, and HalTs (1980) work on implementation configurations 
would be useful as well. 

In any case, as already noted, treating implementation as 
soc'ialization certainly seems to have descriptive utility, along 
with Elmore's models, for providing enlightenment as to what has 
happened in the past, especially the Follow Through past, and 
these are important considerations in filling out this discussion 
of implementation realities* Fullan (1980) also sees two principal 
sets of factors as affecting implementation/socialization. These 
factors — and subf actors — are as follows: 

1. Characterfstics of the change effort 

a. The adoption decision' 

b. Clarity 

c. Complexity 
Implementation strategies 

e. External /internal relationships \ '. ' 

2. Characteristics of the Institutional Setting 

a. History of innovative attempts 

b. Role of central administrators 

c. Role of principal 

d. Organizational characteristics 
eT Community characteristics 

In the following, in an effort to provide further grounding * 
in reality for implementation issues, each of these subfactors will , 
receive brief comment, either with reference to statements in the 
professional literature or from historical Follov/ Through events: 



la. The ad option decision , Hanex (1977a, 1977b) and others 
remind us that little time and little planning were associated 
with initial Follow Through decisions, Mann {1978b) feels iciest 
adoption decisions are unfortunately made on opportunistic bases. 
Evaluation qf NDN (1977) findings give some precise recommendations 
about , e,g,, personal involvement at all levels in the adoption 
decision, 

lb. ClaHt^, The W Follow 
Through appear to have been the most clearly specified. Apart 
from their theoretical base which would stress precise formulation, 
one wonders whether or not Cronbach's (1970) distinction between* 
"maximum" versus "typical" behavior might^e applf cable here also, 
which would imply that the structured approaches were able to be 
more precise — after the fashion of achievement tests —because 
they seemed to continually focus on eliciting "maximum" behavior, 
In contrast to the more unstructured approaches - with parallels 
in personality measures, and their problems, etc,, - which were 
more concemecl with developing "typical" behavior. 

Ic, Complexity . Fullan (1980) tends to feel there is an 
inverse relationship between clarity and complexity,/ McLaughlin 
(1978) ^ on the other hand, in her study of organizational change 
pro.iects, found implementation was not only not Impeded, but 
actually enhanced by complexity. It Is perhaps worth noting that 
Snow (1980) has reconmendations for "demystifying" the concept of 
"learner-control," which has represented quite a complexity for 



sc^j^cf the unstructured models. Child Development considerations 
could si s^ perhaps be made less complex by tending to, e.g., Elkind* 
recoinnendations on using Piaget as a thinker rather than an educate 
(cited by Albin, 1980). . : ' 

Id. Implefiientaticn strategie>^ , Fullan (1980) feels it may be 
counterproductive tb try to-.^lain everything ar once, and that 
it may. be especially important to postpone conslderatio.ns of 

the innovation's philosophical bases until participants are more 

•' ' / 

receptive. ASCD Update (1980) notes that there is widespread 
teacher discontent >^ith support in their ciirriculiCT development 
neet3s. Perhaps Kessen's (1979) rscommeri'dation on seeing child 
development In a cultural context, and Srcufe's (1979) insights 
on the coherence of i-ndividual development should be the kind of 
background context provided for most innovation. ' ""'^f 

le. ' Extem£^l /internal relationships . It is perhaps ironic 
that experience"With the various Follow Through Models' sometimes 
led to the conclusion that the more structured the model, the more 
flexible It was. in accorrmadating local needs and, circumstances, 
and the contrary for the more unstructured, which sometimes, 
appeared to have either taken a very defensive position, ;or else 
indicated that they had nothing 'to prove. 

Sponsor inrertjal stability. and therefore continuity of model 
develppiTient were crucial factorc/ in Follow Throii^ 



-25-- 



^2a. HI s to r y^ of Innovative attempts , Fullan notes that Sarason's 
"'prehistory concept is of great importance here. In large cities, 
especially, jt would seem that Sponsors had to overcome a good deal 
of cynicism or a "show me" attitude because of all the ''panaceas" 
to which these school systems had been subjected since iH§ 50's. 
^ Mann (1978a) notes the primary and complexity of the sites in 
all cases; a complexity that an unfortunate "prehistory" could 
compound. 

' . 2b. Role of central a dministrators . The innovations of 

modem education very often stand or fall with administrators' 
approval /commitment, which is apparently often tied to their 
own apportunities/ami>itions. Some adninistrators , on the other 
hand, might resort-io a "power strategy" (Patterson and 
Czajkdwski, 1979)- 

2c. Role of the principal . Fullan sees the principal's 
supportive role as absolutely essential; he unfortunately has 
concluded that 50-60% of the nation's principals are not 
instructional leaders, but administrators only, and this fact 
often hinders implementation. From another point of view, it 
may turn out that some few individual principals may be able to 
set up a school environment that is perfectly in keeping with all 
the control factors of Elmore's Model I (Systems Management). . 

2d. Organizational characteristics . Dealing with Unions 
/N is a critical matter, for large cities especially; there should be 

concerted efforts at all levels, federal, state, local - to 
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either neutralize any obstacles stemming from existing Union agreements - 
perhaps by concluding a special, separate agreement - or by including 
Union members as essential participants in the original implementation 
decisions so that this element of "ownership" generates cooperation 
throughout the project's duration. Other factors that have had 
considerable effect on implementation have been, e.g. (1) strines, 
especially long, bitter strikes, (2) periodic funding crises resulting 
in lowered morale, (3) school district reorganizations during the year, 
(A) lack of funding for adequate staff development, and (5) conflicts 
with other federal funding sources, such as Title I eligibility denands, 
conflicting program foci, etc. 

2e. i^nmiumtv characteristics . An incident of some relevance here 
is the following: a model with an essential family, outreach component . 
was selected for an area of the city with crime-ri<lden high-rises; the 
• location effectively helped stymie implementation; staff assigned home 
contact responsibilities were simply afraid to carry them out and avoided 
doing so whenever they could. 

Bridge (1978) makes some useful recommendations regarding parent 
involvement, e.g., when and when not to strive for group consensus. 

C. Recommendations f o r Fu turg. Fgllw Thi^^ Implementation Efforts. 

It has already been anticipated 1n earlier ...statements that the 
main tenor of this reconmendation will be that Follow Th rough Mod els 
(Sponsors) move^^yTteadVIyTowI^d not just a "mutual Id^^atTSn" . 
mode, but toward Elmore's Model HI approach, i.e., a highly 
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conmitted organizational development stance that focuses on 
developing "self-starting capacity" in each teacher. (The Follow Through 
Program in Philadelphia has begun moving in this direction on its 
own, first establishing local derivations of models, and combinations 
of models, and more recently initiating a "best elements" approach, 
selecting the best aspects of a number of models.) The 
prir.c1pal implication here is that Sponsors should primarily become 
co-facilitators with site personnel in developing personalized models 
for each teacher — which in turn .implies cross-sponsor cooperation, 
various forms of "hybrid-model" thinking, etc., in order to best 
serve each individual teacher in developing his/her highest possible 
level of teaching expertise. All that has been learned in function- 
ing at the global model level can now be taken the next -step forward 
in a plausible developmental sequence, i.e., personalizing, tailor- 
making the models in a very literal sense, there seems to be a 

.i 

very real mission for Follow Through in this conceptualization, one 
that has to do with close alliances with sites in working toward the 
effective development of highly skilled teachers in compensatory 
education through enabling model -grounded elements to become realized 
in unique, personal, "best- fit" models. 

In this conceptualization, teacher commitment would be built in 
by definition, since the approach presupposes that sites would be 
funded to work with low-achieving schools through enlisting teacher 
support for further. self-development, which in turn would involve 
Sponsor(s) expertise in helping sites accurately diagnose the needs 



(expressed and observed) of each teacher so that self-motivated, efforts 
result, geared toward developing a personalized model for each teacher. 
Each site would have available a panel representing a range of 
structured/unstructured sponsors (realistically, perhaps no more than 
a team of three or four sponsors would be needed) who could work 
cooperatively together and with local training staff in helping each 
teacher develop an effective, unique teaching style based on a special 
combination of best model elements. The approach, of course, still 
relies on some total program coimionalities: small group instruction, 
multi -adult classrooms, parent involvement, etc. 

The above recoirmendation makes one further presupposition about 
the possibility of close Sponsor cooperation, and that is that a , 
flexible, eclectic position be taken vis-a-vis one another, so that 
whatever fundamental philosophical differences may at one time have 
been associated with particular models, these be put aside in the 
interests of developing effective approaches that represent the best 
form of rational cooperation in a world where no one theory/approach 
has proven to be unassailable. 

In the evaluation section below a major research and evaluation 
role will -be recoimended for Sponsors as uniquely abte and experienced 
investigators and, therefore, much in demand in moving the study of 
implementation forward and expanding on e.g.-. Snow's (1977) and 
Cronbach and Snow's (1977) urgings regarding aptitude— treatment 
interaction through special expertise in developing the teacher- 
treatment aspect in the form of personalized models. 



IV. EVALUATION COMPATIBLE WITH IMPLEMENTATION REALITIES 

This final section will attempt to bring together the points 
made above about implementation, both as regards philosophical/ 
theoretical considerations and its more concrete realities, as 
illustrated in the last section, and what was said earlier about 
philosophical/theoretical premises, in the evaluation area, . intending, 
as a result, to produce a concrete proposal for the evaluation of 
implementation- The section will be divided into three parts: 

(a) a general consideration of evaluation strategies compatible 
with the philosophical/theoretical assumptions made earlier, 

(b) special ^.focus on the single strategy considered most appropriate, 
and (c) a final reconmendation regardina the evaluation of 
implementation. 

: A. General Consideration : Evaluation Strategies Compatible with 
Earlier Assumptions 

1. Some Compatible Approaches 

As a general introduction to the approaches to be listed 
below, it might be useful to give prominence to Good! ad's 
(1979) two reconniendations on research into classroom management, 
since they reflect the basic orientation of many of the strategies 
listed: 

The first is that conclusion-oriented researchers 
interested in understanding classroom phenomena 
get involved in the "stuff" of practice — in 
naturalistic, ethnographic inquiries, • .The second 
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suggestion is that decision-oriented researchers 
interested primarily in improvement should 
collaborate with teachers in inquiries in which 
assumptions and purposes are shared. 

(Goodlad, 1979, p. 407) 

The compatible strategies include, among many other 
candidates: Blumer's (1970) symbolic interaciionist outlook 
with emphasis on direct "exploration" and "inspection" of the 
"empirical world"; Carini's (1975) "descriptive research"; 
Bronfenbrenner's (1976) "ecological experiment" approach; 
Busses, Chittenden, and Amarel's (1978)" collaborative research"; 
Bateson's (1980) insistence on the primacy of description versus 
our usual predilection for sanctioning "facts" only after process- 
ing in the "magical" realm of explanatory mathematical models ;- 
Cuba's (1978) general endorsement of, natural iis'tTc versus 
conventional inquiry; Stake's (1978) recommendations on case 
Studies and naturalistic generalizations; Eisner's (1979) -f^' 
proposals for effective use of an "educational criticism" strategy; 
Weiss' (1980) suggestions for turning our attention to- 
"nonconventional" outcomes of schooling, and their assessment, 
and Kean and McNamara's (1978) "gradual refinement" approach, 
possibly with Apple's (1980) additional- recofmiendations. 

A Brief Reflection on Measurement 

In leading up to the final recommendation, it will serve a 
useful purpose to briefly reflect on 'the nature of educational 
measurement. Jones (1971) reminds us: \ 
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Some measuren^nts can be made quite sinqjly and 
directly,.. Other attributes, however, can be 
estimated only from their effects. ..^ The inference 
about an attribute from its effects involves either 
an assumption or a demonstration of a relation 
between effect and attribute. 

(Jones,. 1971, p. 337) 

We often forget that all of our educational measures involve 
inferences from effects. ' The article fay the Laboratory of 
Comparative Human Cognition (1979) quotes Goodenough to much the 
same effect, noting that our educational and psychological measures 
are not "measuring devices, properly speaking. They are sampling 
devices." Cronbach (1971), and Messick (1975, 1980) have always 
stressed the prime importance of construct validity precisely 
because judgment/inference is at the heart of our measurement 
process. Cronbach (1970) sees construct validity as the Hong - 
continued interplay between qbservation, reasoning and imagination, 
and Petrinovich (1979) likens the entire process of scientific 
inquiry to the search for construct validation. The main paint 
is that perhaps we have been going in the wrong direction in our 
attempts to better our instruments; it seems reasonable to arguie 
that bettering pur measures is essentially a matter of refining 
our judgments, making our inevitable inferences much more 
sharply focused and competent. 

The Need for Model Sponsors in Evaluation / . 

Evaluation, especially the evaluation and study of 
implementation is one of the most important functions Follow 
Through Sponsors can provide. Not only are they now uniquely 
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experienced in the struggle to deal with implementation of their 
models, but they also have all the advantages, e.g., Gallagher 
(1979) attributes to research centers, i.e., situations allowing 
long-term coirmitraent to instrument development, sustained and 
organized concentration on major problem dimensions and their 
interrelationships, and the possibility of being in a position to 
discover the unexpected, important insight. Follow Through 
Sponsors are an irreplaceable third party unit in this paper's 
conceptual. Uation of Follow Through 's future because of their 
dual capabilities as unique co-facilitators in the development 
of personalized models, as already mentioned, and as uniquely 
equipped evaluators of the complex Implementation process. 

The Single Most Compatible Approach : Evaluation as Judici al Hearin_a 
Analogue. 

Wolf (1979, 1980) is, as is well known, one of the chief 
proponents of the Judicial Evaluation Method (JEM): 

the law, as a metaphor, offers many Important concepts 
(fact-finding, adversarial proceedings, cross- 
examination, evidentiary rules and procedures, structured 
deliberations, etc.)that when adapted to evaluation efforts 
add certain dimensionality lacking In more conventional 
forms of social Inquiry... The ultimate evidence, then, 
which guides deliberation and judgment Includes not only 
the "factSj" but a wide variety of perceptions, opinions, 
biases, and speculations, all within a context of values 
arid beliefs. Oftentimes the more subjective forms of , 
evidence help put the facts into proper perspective. 

(Wolf, 1979, pp. 20.-21) 



He is aware of the many criticisms leveled against the process, but 
but replies that most of these come from misunderstanding the intent. 



and suffer from taking the "judicial" in literal, rather than * 
metaphorical form. One of the most recent criticisns is that by 
Worthen and Rogers (1980). They feel the approach - which they 
label "adYersaT7," an identifier continually avoided by Wolf — 
is not useful for formative decisions' and is "most appropriate 
for large, controversial programs which had a variety of 
interested audiences" (Worthen and Rogers, 1S80, p. 540). It 
is important to remember, however, that the judicial "metaphor" 
is the main point, and that with Thurston (1978), for instance, 
several other judic^vl forms than the jury trial are available 
and may be more useful on occasion, e.g.,, the appellate court 
model and the administrative hearing model. It is this paper's 
position that the implementation process being the evolutionary, 
multi -faceted phenomenon that it is, 1 involving conceptual changes, 
reconciling of organizational roles, etc., this is exactly where 
fixed measures are least able to cope, and the judgmental base 
of all measures becomes most important, so why not formalize this 
importance through use of some judicial or quasi-judicial method? 
These methods can range from some of the peer review processes 
suggested by ApUng (1979) to the fullest kind of parallel with 
a judicial process. "The basic procedure is the crucial .considera- 
tion, and can be made td\involve enough stages and safeguards to 
insure against premature '°final judgments," while at the same time, 
-keeping^deastons^out^iji^the^open^jrequently^en^^ 
postponing those critical go/ no gb^ decisions when they sfr^ould 




\ 
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be made. Walker (1977) indicates that Schaffarzick: 



suggested that greater efforts at systematic 
consideration of curriculum .change may be 
more likely when the community context is more 
aroused and polarized on an issue than- on more 
routine issues because the arguments and 
evidence resulting from systematic exploration 
of the issue may be more effective persuaders 
in such a climate. 
>r Offalker, 1977,. p. 295) '' 

The pudicial analogue approach has immediate relevance and usefulness 
in this same vein^. because it allows the teacher participants in 
implementation to have a forim for settling what may be very real 
problems in dealing with the required changes of a specific 
innovation. 

C. Basic Conclusion/RecofTinendation ; Combined Documenting and Judicial 
Evaluation Approaches 

The paper began with a systems theme, and it is appropriate 
that it end on that theme. {Note that "systems" here has nothing 
in coninon with Systems Analysis (Systems Management.) Messick (1972) and 
Messick and Barrows (1972) have stressed the. need for comprehensive 
documentation and regard for the fact that we are continually - 
dealing with varying forms of systems in education: 



To recapitulate briefly^ it looks as if the 
nature of the thing to bB changed, the educa- 
t1onaJ_jirtha,. Js — ~~ 

composed of many constituents but a complex 
culture comprising multiple roles and pi ura- 
Usticj/alues, and that if we are to understand 
the functibning of that" system /•^"'^or'^tb'^^^^^ 
it — we must take into account the interplay 
of those roles and values in supporting (or 
subverting) system regularities. 

^ (Messick, 1972, p. 79) 
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McNamara (".980) reconinended that comprehensive, computerized 
networks of basic school experience data (for longitudinal tracking) 
— that would translate here into implementation data — be 
developed by sites that would serve as a "skeletal" structure to be 
continually flushed out by ongoing case studies, participant observati 
and collaborative research. The Research and Evaluation Conmittee 
of the Follow Through SCAN Forward Planning Task Force* has 
reconir^^ Through Data Base (Appendix), Hanson and 

Schutz (ia7a)-hpepo rt^-^a-%r^^ approach, and Cooley and 

Leinhardt (1980) have recently concluded an instuctional dimensions 
study within a similar framework. These are the kind of 
documenting systen^, which, when combined with various types of 
judicial hearings at regular intervals — perhaps 3 times a year 
formally, but once a\month in inpre informal fashion, e.g., in the 
form of reviews, etc.^r- would appear to be a sensible way of 
capturing and constantly contributing to the improvement of the 
important, evolutionary process of Implementation. JThis is felt 

to be particularly true of Follow Through implementation, whether 

\ ' 

this remains a matter ojf only^some mutual adaptation on the part 
of sponsar*^&it3<1br, what is considered the ideal situation in this 
paper, represents a steady move toward directly personalized models. 

\ 



♦Members: M. Wang, T. Gennarl, J.'Lurrt)l|)y, S. Fox, J. Rogers, 
D. Revicki, E. E llett and T. McNamara \ 
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A NATIONAL 
FOLLOW THROUCIH DATA BASE " 
RATIONALE 

Introduction 

A new <llrectlon is set for Follow Through, to take effect In 1981. 
No longer is the program officially targeted for dismantling by the federal 
government? instead it is nov sanctioned as a program with two different, 

though complementing missions. Its first mission essentially conf inues the 

■ - .f . ■, .. 

original service goal of the program, though, as previously, the program 
even In tSls respect is not conceived in purely service terms. There is aljo 
new- explicit emphasis on ^naglng this uniquely comprehensive educational 
effort to a degree of effectiveness that makes this nominally' "service" 
operation a de facto demonstration program. Eighty percent, of the sites 
nationally are scheduled to Be this kind of Service Project. 

Under its second mission, Follow Through is now to engage in pilot 
Knowledge Production activities in 20% of its sites. In this respect the 
program takes on a specific research resource dimension that can simultaneously 
provide consistent Improvement for its .demonstration CServlce) commitment to 
the education profession^' and continual growth and new development * in basic 
understanding of the teaching/learning phenomenon in early childhood. ^ 

Given this new, two-fold direction for Follow Through — and remembering 
its highly controversial history, both programmatically and in terms of evalua- 
tion, — It appears to be an Important consideration to provide a mechanism 



that will allow this multi-faceted program to be continuously documentable 
In all major respects, and at the same time to be. transformed via this doc- 
umentation into readl"*y. available configurations of variables that can be 
meaningfully researched and evaluated. A irrational Follow -Through Data Base 
is proposed to sexrve as this essential mechanism, 

. The concept of the 'National Tbllow ^ Thrbugh Data Base will be fully ex- 
panded in the sections below. By way of further clarification of why. it. is 
proposed, the immediately following comments "are offered regarding the kinds 
of information that are needed for this program, as well as Its res^earch and 
evaluation needs. 

■ >Klndg of rnfonaatldti Needed 

Follov Through is a highly complex program, and even more complex now in 
Its "new look." First It *s a *^nultld±mensional" program. That is, it serves 
as a "natural resource," demonstration program CServlce aspect), .'for early 

childhood development and education" through- ^our program components (education, 

- ■ ■ " . • ■ . ' . . . ■ ■ . . 

parent Involvement, staff development, and support services) . Follow Through, 
furthermore, emphasizes not only service related to these four components, 
and planned variation^, but their evaluation as well, and now In its ICnowledge 
Production role also has a clear-cut research responsibility. 

A second characteristic of Follow^ Through in its Service capacity is 
that it is still an experiment in planned variation. As a result. Follow 
Through continues to include a diversity of Educational models. Although all 
are aimed at promoting early childhood development and' education, the models 
^differ in p'hllosophical and theoretical orientations and in the specific 
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approaches thsy employ. \ 
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In addition te the workings of each of , the four general progrnm components, 
the contribution of each of the models represented In Follow Through must be 
recognized In any documentation/evaluation effort for the overall program. 
However, providing Information only on the mode of operation and effectiveness 
of each model Is Insufficient. Personnel in the United States Dept. of Education 
must be. able to present Congress with data about the overall impact of the pro- 
gram for Tise In deliberations regarding future authorizations. Thus, evaluation 
efforts must Include aa emphasis on the program as a whole in relation to each 
of the four component*, as well as preserve a correct perspective on each 
sponsor's contribution within each, component. 

- The multidimenslonallty and diversity of the Follow Through program 
suggest that at least two kinds of documentlng/efvaluatlve information ai'fc 
needed under its service aspect. First, there must be broad-based information 
that is obtained by all sponsors at each participating site. This informa- 
tion will ' useful in assessing and, ref ini.ng the service that Follow Through 
provides, and then will in turn feed the Knowledge Production aspect by 
helping to identify potentially beneficial efforts in the areas. of early 
childhood ■ development and education, and, more specifically, in generating 
innovative evaluation/research strategies for studj.ing the impact of a wide 
range of - interventions in a variety of contexts. 

A second kind of evaluative Information required for the program's 
main function is Information related to the needs and Interests of particular 
sponsors and participating sites. This information is crucial in a national 
evaluation of Follow Through service j But at the same time it must be 
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recognized that Its prime purpose Is to contribute to the assessment and 
refinement of specific models In their service capacity, and to add to 
existing research knowledge regarding educational Innoy^tion (Knowledige 
Production dimension) . ^ 

Specific information related to the needs and interests of particular 
sponsors and sites catihot be provided in this proposal. Eowever, recommend 
atlons can be made regarding the kinds of information that should be gathered, 
or at least furnished, by the total program, i.e., ncrogg all sponsors. Five 
tjrpes of data are of particular importance; and would bi2 of major concern in 
the construction' of g- Nationsl FollQXjr Throughr Data fase: , 

a. Site characteristics 

b. Program characteristics 

c. Delivery system characteristics 

d. Degree of lanplementatlon 

Jim Outcomes i 
Each type of information is brief ly discussed below: ' 

a. S ite Characteristics 

Previous studies have suggested that individual sites differ 

^ -= ■• : 

radically- in terms of a variety of variables, e. g,, student 
' .' ^ ) " ■ ' . . ■ ■" -. - ■ 

characteristics, staffing, school and community setting. Data 

; / . : • ■ ■ • ■ , ■ : . . . 

should Be gathered on these variables for possible use In ex- 

I ' ■ . ' ' . - t- ■ ■ -. ■\ ■ ' • ■ ■ • 

plaining some of the outcomes that are achieved at particular ^ 

I ' ■■ ' . * ... ' . ■ . . - ' ■, * - ' 

sites'. Variables that are found to be significantly related 

to outcomes can then not only remain th6. concern of evaluation* but also 



be a focus of future research efforts • 



b. Program Chardct^rlgtrlcs . \ 

At least two characteristics, of the model programs designed and 
Implemented by Follow Through sponsors are of particular inSportance . 
These characteristics are: objectives and critical features! Most of 
the programs have the same general objectives, b^yt tiiey differ -in 
the Importance they assign to each and in the specific approaches used 
to meet these objectives. Information on objectives and approaches can 
be used In interpreting evaluation results. As an example, 
of low scores on measures of personal and 'social development 
minimized for a program that emphasizes basic skills. Instekd, the pro- 
gram would be judged primarily on outcome data related to thL basic 
skills objectives that it sought to attain. 

c. Delivery System Characteristics 



^he Importatice 
would be 



Model programs differ not only in their objectives and [features, 
but also in the delivery systems they employ, e.g., number of sites, 

• ■ " ■ ■ ' ' i . 

■ 

Structure of delivery system, staff involved, etc. 

A National Sata Bast: should also take this t3rpe of program 
dlvezsity into account and include the collection of data^bn character- 
istics of the delivery systems implemented by various Follow Through 
sponsors. 



d. Degree of Implementation 



The need for including Implementation data has become increasingly 
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apparent In relation to Follow Through,. There are at least three 

' • \' ■ 

reasons why such Information Is essential. First, sponsors have to 

\ 

clearly define their models. These definitions should then bv^xorae 

^ ■ \ . 

operational so that relative judgments about levels of model Im- 
plementation across sites and across schools or classrooms (or even 
families, for some models) within sites can be made. Inability to . 
produce Implementation scales in some cases may be due to lack of 
clarity in the definition ''of the model. Such a lack of clarity 
probably foreahadowa future diffici3lties. 

Second, Implementation scales are needed to provide Information 
that permits the linking of outcome data to implementation levels. 
Some sponsors have already begun using implementation data in their 
evaluation designs. Prom the sponsors'* perspective, ^uch designs are 
superior to designs concerned only with outcomes, since they allow 
some.separatlon of model. characteristics from local site variations 
and, hence, represent truer tests of th« model itself. 

A third reason for* good implementation data stems from an interest 
in implementation as an outcome Itself. For some sponsors, changes in 
the educational process represent key outcomes of the sponsor intervention 
apart f rom ai^^^ncerh with products such as test scores. It appears 
important that interest in implementation ns an outcome should not be . 
restricted to these sponsors alone « 

There are some further important aspects of Implementation that 
should be noted at this point in view of a National Data Base effort: 



In evaluations of sponsor delivery systems, Implementation variables 
will be the most direct Indices of delivery systems ef fectilveness. Such 
evaluations should be valued for formative purposes as well as externally 
If Follow Through is truly to live up to Its potential as a "natural 
resource", both for donoustration and research purposes. 

For wlthin*<»sponsor comparisons. Implementation ratings need to be 
reliable and comprehensive, but can be model-^speclf Ic. For_ across*S3ponsor 

comparisons, there la a need for Implementation judgments that have some 

»f < . . 

cojTOon meaning for all sponsors involved, atr least to the point of quanti- 
fying the meanings of "Xow-;,^ "moderate,'* and "high" levels of Implementa'- 
tion. An approach to the development of Implementation scales is proposed 
in the section on Implementation below. 

e. Outcomes - 

To date^ an almost exclusive focus in determining the effectiveness 
of Follow Through has been student achievement. However, given the . 
multldlmenslonality of the program, it is critical that data be gathered 
on other out comes. related to the education component (e.g. , student 
autonomy, ^eif'H^oncepti, as well as to the other three components of the 
national program. Examples of outcomes that might be considered include:- 
quality of school eirvironments, student/staff interactions, and community 
involvement. As noted earlier, ^odel implementation may also be viewed' as 
another outcome^ as. may some .specif ic site characteristics, e.g., ability 
to transform family status. 
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Evaluation /Research Strategies Needed 

Once decisions are made regarding the specific kinds of information that 
are required fm order to meet the information needs of various stakeholders 
in tbe Follow Through program, evaluation/riesearch strategies for ccAlecting 
and analyzing l.Viii information must be idetiilfied. Two issues related to such 
strategies seem of particular significance. The first concerns the need for 
alternative strategies, the second with program change. 

As indicated in the previous section, a wide variety of "^information is re- 
quired in order to study and assess a program as complex as Follow Through, 
One large-scale experimental study such as was conducted previously cannot 
•possibly provide the inf ormatiQu^^equired. In planning a National Data Base 
for the new Follow Through, alternative designs, measures, and analysis pro- 
cedures srmld be identified anfi the most appropriate ones selected or developed 

In terms of alternative designs, any soundly constructed design that offers 
the opportunity of -increasing existing knowliadge should be considered. This 
approach, while limited by economic constraints, would open up Follow Through 
to much more innovati-ve documentation, and research. These designs could con- 
tribute significantly to the entire field of educational investigation, as well 
as furnish information that would be beneficial to program stakeholders at the 
local, sponsor, and national levels. 



Of particular interest at this time is a combination of quasi-experimental 
and descriptive research. Descriptive data cannot only serve as a complement 
to data gathered through experimental methods, but serve as useful investigative 
means. in their own right. Descriptive data collected on classroom processes. 



for example, can prove invaluable in generating research questions for 
field experiments aimed at refining program coiaponents. -Further, these 
data are useful in accurately documenting the process and outcomes of 
particular programs under certain specified conditions. 

The second major issue related to evaluation/research stratesies con- 
cems program Chang ». During the course of any given year and particularly 
from year to year, if the past is any indication of the future, there will 
be changes in the Follow Through program at all levels. National objectives 
may be revised, objectives of particular model programs may be refined, 
and new delivery systems for Implementing programs at participating, sites 
may be. developed. Thus, it is critical that evaluation/research strategies 
are capable of providing documentation of such changes. Further, if certain 
strategies are deemed inappropriate, given program changes, then these 
strategies must be modified or replaced by strategies that can more 
effectively meet program needs, and documentation that can handle, this 
contingency should be available. 

These strategies are based on the assimiption that the "new look" 
Follow Through Program will build in adequate funding from year to year 
for research and evaluation in the service dimension as. well as in the 
Knowledge Production area, and that there will be continual give-and- 
take in research and. evaluations ideas and proposals between sponsors,. 
- sites, and -the Dept. of^^Education. 

o 
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DATA BASE 



PRINCIPAL ELEMENTS OF THE FILE 

The basic concept of a National Follow Through Data Base asstimes a 
common computerized file creation effort across the program. This common 
effort would principally revolve around the development of two types of files 
to be maintained at each site: a) a child data file and (2) a site 
descriptor file. In addition each sponsor might find it profitable to main- 
tain a separate file system that contains key elements of the child data and 
site descriptor files in existence at each of the fyponsor's sites; the sponsor 
systems should probably also include the kinds of delivery system character- 
istics outlined in Table 1 below: 

1. CHILD DATA FILE ELEMENTS 

A- Fixed Length Portion of File 

1- ID - cotamon ID approach across all sites — basically a ntmeric 

system with initial alphanumeric character to di5;tinguish. site. 
2* NAME fixed as well as phonetic spelling -p-r the latter for better 
tracking later on, in case of use/disuse of initials, etc, 

3. BIRTHDATE 

4. RACE CODE _ 

5. SEX COTE — 

6. SES INDICATORS - e;g. , perhapu median income/% on welfare by site, 

geographic areas, etc. 

7. . PRE-K EXPERIENCE CODE - basically yes/no; perhaps type 



8. ENTEimiG K/FIRST GRADE TEST DATA ^ essential pre-program da>.a 

to Be der±^7ed from common instrument/subtest across 
all sites, 

B. Variable Ltogth Portion of File 

N.B. For each year after the child enters the pr ogram : 

1. SCHOOL YEAR 

2. SCHOOL CODE 

3. GRADE 

4. CLASSROOM IDENTIFIER 

5. LENGTH OF PROGRAM EXPOSURE (also model indicator 
in XB\ilti--model sites) — e.g., in months 

6. YEARLY ABSENCE RATES 

7. SPECIAL EDUCATION/GRADE REPEAT INDICATORS 

8. LOCAL SITE STANDARDIZED NORM-REFERENCED ACHIEVEMENT 
TEST SCOPES CPoEsibly better managed by pre-post testing 
on a matrlz-^sampling basis using a common instrument 
across all sites) 

9. SPECIAL SPONSOR TEST DATA 
10. CRITERION-REFERENCED COGNITIVE/NON-COGNmVE TEST DATA 

CrO BE DRAWN. FROM A COMPREHENSIVE' ITEM POOL TO BE ESTABLISHED 

FIVE YEARS AT THE LATEST) 

II. SITE DESCRIPTOR FILE 

A. Fixed Length Portion of File 

1. SITS CODE (As in CHILD DATA FILE) 
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1. BASIC PROGRAM DATA— e,g., # Schools, ? Teachers, ;^ Aides 
# Children, Funding Sources, etc. 
B. VariaBle length Portion File 

1. SCHOOL YEAR ' 

2. CROSS-PROGRAM, DEGREE OF IMPLEMENTATION DATA BY SCHOOL, 
GRADE, CLASSROOM # large representative saiaple each 
year; using 'Tiigh/' '^nedlum," "low" indicators relative 
to ideal implementation 

3. SPECIAL SPONSOR IMPLEMENTATION DATA 

4. SPECIAL SITE IMPLEMENTATION DATA (OPTIONAL) 

5. SUMMARY TEST DATA DERIVED FROM CHILD DATA FILE; 

6. PARENT INVOLVEMENT INDICATORS — e.g., some basic data 
regarding i? yoltinteers per family, # volunteer hours, 

# workshops attended, # cither activities attv»nded 

7. STAFF DEVELOPMENT INDICATORS' — e.g., some basic data 
regarding general staff experience, experience in program, 
9 consultations received, # workshops attended, etc. 

8. SUPPORTIVE SERVICE INDICATORS-^—' e.g., some basic data 
regarding i? -medical/dental screenings, # treated; 

# psychological referrals, # treated: # families 
contacted^, # in need of social services, # being 
helped through, social services 

9. SPECIAL "NATURALISTIC INQUIRY" INDICATORS ~ e.g., 
possibly special status Ctotal site/specific aspect) 
codes derived from participant observation » in-depth 
interviews^ questionnaires, etc,, describing some 



59 



"emarging" program characteristics as it functions in 
the real xjorld; maybe there - should be some cross-program 
arrangements for continuous, representative efforts at 
participant observation 

The site descriptor file could be further specified by taking into 
account the list of variables appearing in Table 2. 

Table 1 

£z£imples of Delivery System Characteristics 

1. Number of sites served 

2. Total number of people involved in delivery system at the sponsor level 
(a) number of f ull-^tlme' personnel 

Cb) nimiber of part--t±me personnel 

Cci nimiber of people responsible for delivery of services to individual 
sites (person/site ratio) 

3. Total number* of pisople involved in delivery system at the sltec lc rel 

4. Organizational structu«j>f delivery system at the sponsor level 
(e.g., types of personnel, types of direct and indirect service) 

5. Organizational structure of delivery system at the site level 

6. Time spent /(in terms of delivery services) in contact with individual 
sites per' year 

7. Types of interactions (on-site interactions and sponsor-shop inter- 
actions) that take place as part of delivery system 

8. Tjrpes of sponsor/site evaluative feedback processes utilized 
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Table 2 

Examples of Site Characteristics 



1. Students 

(a) mobility 

(b) age 

(c) sex 

(d) ethnicity 

(e) participation in program Cvolimtary/involuntary) 

(f ) previous education Cnursery school) 

2. LEA staff 

Ca) age 
Gb) sex 

Ccl ethnicity , 
Cdi experience 
Cel education 

3. School district/school 

Cal geographic distribution 

CBl # of years in FT program 

Ccl comprehensive ^rvlces offered 

Cd). district organization/structure Cxelation of FT director to 

district organizational structure) 
(e) administrative organization ' 
(fl extent/use of local resources 
Cgl leakout to NFT 
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(h) distribution of FT classrooms vs. entire school district 

(i) additional federal program participation with FT population 
(j) proportion FT/district budgets 

(k) distribution. of sites (re: sponsor and sponsor location) 
(1) sponsor/site "marriage" reasons (who, how, why) 
(m) sponsor/site "divorce" reasons 
(n)-suimner /after-school activities 

(o) proportion of exceptional children in FT classrooms 

(p) retention statistics 

(q) desegregation plan ^process tised) 

(r) union participation 

Cs) assignment of classrooms, to Follow Through (voluntary/involuntary 

Community 

Cal rural/urban 

Cb). population * 
Ccl SES 
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